State Generation and Automated Class Testing: a Case Study

The maturity of object-oriented methods has led to the wide availability of container classes: classes that encapsulate classical data structures and algorithms. Container classes are included in the C++ and Java standard libraries, and in many proprietary libraries. The wide availability and use of these classes makes reliability essential, and testing plays a key role in achieving that reliability. The large number of cases necessary for thorough testing of container classes makes automated testing essential. We present a novel approach for automated testing of container classes based on combinatorial algorithms for state generation. We illustrate the approach with one white-box and two black-box test drivers for a class implemented with the red-black tree data structure, used widely in industry and, in particular, in the C++ Standard Template Library. The black-box driver is based on a new algorithm for red-black tree generation. All three drivers are evaluated experimentally, providing quantitative measures of their e ectiveness in terms of block and path coverage. The results clearly show that our approach is a ordable in terms of development cost and execution time, and e ective with respect to coverage achieved. The results also provide insight into the relative advantages of black-box and white-box drivers, and into the di cult problem of infeasible paths.